Computer implemented method and system for computing and evaluating demand information

ABSTRACT

Computer implemented method and system for improving demand forecasting by estimating the hidden demand at an occurrence of a sellout using a single parameter probability distribution with a parameter assuming a forecasted mean demand value derived from a statistical seasonal causal time series forecasting model of count data on a new data set of sales values excluding truncated sales values at occurrences of sellouts. The present invention also provides for new more accurate performance evaluation techniques together with new performance metrics for evaluating an actual draw and for comparing a recommended draw to an actual draw.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of and claims priority from U.S.patent application Ser. No. 10/098,615, filed Mar. 18, 2002, which isincorporated by reference.

FIELD OF THE INVENTION

The invention is in the field of computer implemented methods andsystems for computing and evaluating demand information in general, anddemand information for perishable consumer items based on sales dataincluding sellouts in peculiar.

GLOSSARY OF TERMS

The following terms listed alphabetically together with their acronymsare employed in the description and claims of the present invention:

Adjusted Sales Data

Sales data with expected full demand (EFD) values replacing truncatedsales values (S) at occurrences of sellouts.

Availability

The probability of fully satisfying the demand for a consumer item at anoutlet under a given draw or, in other words, satisfying demand for theconsumer item at the outlet without an occurrence of a sellout due todemand being at least equal to draw.

Distribution Policy

Delivered quantities of a consumer item at each outlet of a supply chainsupplying a multitude of outlets in accordance with a predeterminedbusiness strategy.

Draw (D)

An industry term referring to the delivered quantity of a consumer itemto a particular outlet under a given distribution policy.

Draw Bounded Sales (BS)

Sales quantities for a consumer item which would be nominally recordedon the assumption that the consumer item is delivered in accordance witha recommended draw instead of an actual draw i.e. sales quantitiesdisregarding hidden demand at occurrences of sellouts.Draw Bounded Returns (BR)Return of a consumer item which would be nominally recorded on theassumption that the consumer item is delivered in accordance with arecommended draw instead of an actual draw i.e. returns disregardinghidden demand at occurrences of sellouts.Draw Bounded Sellout (BSO)A sellout of a consumer item which would be nominally recorded on theassumption that the consumer item is delivered in accordance with arecommended draw instead of an actual draw but disregarding hiddendemand at occurrences of sellouts.Draw Bounded Stockout (BST)A quantity of unsatisfied demand which would be nominally recorded onthe assumption that the consumer item is delivered in accordance with arecommended draw instead of an actual draw i.e. quantity of unsatisfieddemand disregarding hidden demand at occurrences of sellouts.Expected Full Demand (EFD)The sum of the truncated sales value at the occurrence of a sellout of aconsumer item at an outlet and the estimated hidden demand for theconsumer item at the outlet at the occurrence of the sellout, namely,EFD=S+H.Expected Returns (ER)Returns of a consumer item which would be recorded on the assumptionthat the consumer item is delivered in accordance with a recommendeddraw instead of an actual draw taking into account hidden demand atoccurrences of sellouts, namely, ER=RD−ES, for comparison to returns (R)due to an actual draw.Expected Sales (ES)Sales of a consumer item which would be recorded on the assumption thatthe consumer item is delivered in accordance with a recommended drawinstead of an actual draw taking into account hidden demand atoccurrences of sellouts, namely, ES=min(S+H, RD) for comparison to sales(S) due to an actual draw.Expected Sellout (ESO)A sellout of a consumer item at an outlet which would be recorded on theassumption that the consumer item is delivered in accordance with arecommended draw instead of an actual draw taking into account hiddendemand at occurrences of sellouts for comparison to a sellout (SO) dueto an actual draw.Expected Stockout (EST)Quantity of unsatisfied demand for a consumer item at an outlet at anoccurrence of an expected sellout which would be recorded on theassumption that the consumer item is delivered in accordance with arecommended draw instead of an actual draw taking into account hiddendemand at occurrences of sellouts for comparison to a stockout (ST) dueto an actual draw.Hidden Demand (H)The unknown stockout value for a consumer item at an outlet at anoccurrence of a sellout.Mean Demand ValueThe expected value of demand for a consumer item at an outlet thatoccurs at a given point of time.Perishable Consumer ItemA consumer item with a limited shelf life at the end of which it losesmost, if not all, of its consumer value, and which is typically notreplenished to prevent an occurrence of a sellout. Perishable consumeritems can include perishable goods, for example, fruit, vegetables,flowers, and the like, and non-perishable goods, for example, printedmedia publications, namely, daily newspapers, weeklies, monthlies, andthe like.Recommended Draw (RD)A recommended draw for a consumer item at an outlet for comparison to anactual draw. The recommended draw preferably is arrived at byforecasting demand and adding safety stock to provide a predeterminedlevel of availability for the consumer item at the outlet.Returns (R)An industry term referring to the number of unsold copies of a consumeritem on non-sellout days, namely, R=D−S.Safety Stock (SS)An industry term referring to the number of extra stock of a consumeritem above a demand forecast required to provide a predetermined levelof availability. The safety stock for a consumer item is typicallyrounded to the nearest integer.Sellout (SO)An industry term referring to an occurrence of demand being equal orgreater than a delivered quantity of a consumer item at an outlet,namely, SO=δ(D=S) where δ is a binary indicator function:

${\delta({condition})} = \{ \begin{matrix}{1,} & {{if}\mspace{14mu}{condition}\mspace{14mu}{is}\mspace{14mu}{true}} \\{0,} & {else}\end{matrix} $Stockout (ST)The quantity of unsatisfied demand for a consumer item at an occurrenceof a sellout. The stockout for a perishable consumer item is typicallyunknown and therefore stockout and hidden demand for such a consumeritem are the same, namely, ST=H.

BACKGROUND OF THE INVENTION

One computer implemented approach for computing demand forecastinformation for a demand forecast application involves defining aso-called demand forecast tree capable of being graphically representedby a single top level node (00) with at least two branches directlyemanating therefrom, each branch having at least one node, for example,bottom level node (11) (see FIG. 1). The demand forecast information iscomputed on the basis of time series of observations typicallyassociated with bottom level nodes by a forecast engine capable ofdetermining a mathematical simulation model for a demand process. Onesuch forecast engine employing statistical seasonal causal time seriesmodels of count data is commercially available from Demantra Ltd,Israel, under the name Demantra™ Demand Planner.

One exemplary demand forecast application is the media distributionproblem, namely, determining the number of copies of a daily newspaperto be delivered daily to an outlet to minimize two mutually conflictingindices commonly quantified for evaluating the efficacy of adistribution policy for a newspaper over an evaluation period: thefrequency of sellouts, and the number of return typically expressed inpercentage terms of total returns over total draw. In this connection,it is a common practice in the industry that a draw for a newspaper atan outlet for a given day is greater than its demand forecast at thatoutlet for that day so to reduce the probability of a sellout but withthe inherent downside that the probability of returns is greater. In thecase of distribution systems for newspapers, the safety stock istypically intended to provide a level of safety of around 80±10% for agiven probability function for the demand for the newspaper at theoutlet.

The media distribution problem is a particular realization of thewell-known single period stochastic inventory problem which has been thesubject of considerable academic interest. It has been long recognizedthat occurrences of sellouts downwardly bias demand forecasts due toactual sales data reflecting stock availability levels as opposed totrue demand. In view of this, researchers in the area of demandforecasting have developed procedures to cater for presence of selloutsby computing demand forecasts on adjusted sales data. One exemplarapproach is set out in an article entitled “Forecasting demand variationwhen there are stockouts”, Bell, P. C., Journal of the OperationalResearch Society (2000) 51, 358-363.

SUMMARY OF THE INVENTION

Conventional approaches for demand forecasting are based on theassumption that demand for a perishable consumer item is a stationarystochastic process, thereby rendering the estimating statistical momentsof demand from historical sales data possible. Along these lines,conventional approaches assume that demand for a perishable consumeritem may be successfully modeled by a Normal distribution, and thatsmoothing procedures may be employed for estimating demand atoccurrences of sellouts to replace truncated sales values. Against this,the present invention is based on the notion that the demand process fora perishable consumer item at an outlet has a random but non-stationarynature, and thereby cannot be subjected to ensemble inferences based ona single realization thus negating the use of statistical moments toestimate parameters. Hence, in accordance with the present invention,the mean demand values for a perishable consumer item at an outlet overtime are presupposed to be the outcome of a stochastic process which canbe simulated by a forecast model whilst actual sales for the perishableconsumer item at the outlet over time is a particular realization ofsuch a process but upper bounded by the draw at each point in time.Moreover, demand at a future point in time is a random variable with aconditional probability distribution conditioned on the forecasted meandemand value at that point in time.

Based on this realization, the present invention proposes that hiddendemand for a perishable consumer item at an occurrence of a sellout bepreferably estimated by a single parameter conditional probabilitydistribution whose parameter is the forecasted mean demand valuedetermined by a forecast engine employing seasonal causal time seriesmodels of count data commercially available, for example, from DemantraLtd, Israel. The advantage of this approach is that it takes intoaccount important information, for example, monthly and weekly seasonalvariations, predictable and sporadic events, and the like, which isliable to be overlooked if such an approach is not employed. Moreover,it has been empirically found that demand for a wide range of perishableconsumer items at an outlet is adequately modeled by a random variable Xwith a Poisson (λ) conditional probability distribution conditioned onthe forecasted mean demand value λ, whereby the hidden demand at anoccurrence of a sellout is given by:

$ {H =  {E( {X - D} )} \middle| {X \geq D} } ) = {{\lambda( {1 + \frac{f(D)}{1 - {F(D)}}} )} - D}$where f(.) is the Poisson probability distribution function, F(.) is thePoisson cumulative distribution function, and λ assumes the value of themean demand value. Alternative single parameter conditional probabilitydistributions may include inter alia exponential, geometric, and thelike. It should be noted that the approach in accordance with thepresent invention precludes negative demand values which areinconceivable but theoretically possible in the case of the hithertorelied upon assumption that demand can be modeled by a Normaldistribution.

The present invention employs hidden demand not only to improve theaccuracy of demand forecasting for perishable consumer items but also togenerate important new performance metrics for more accuratelyevaluating the efficacy of a draw for a perishable consumer item at anoutlet including inter alia estimated stockout over an evaluationperiod, ratio of estimated stockout to draw, and the like. Moreover, thepresent invention employs hidden demand for more accurately comparing arecommended draw for a perishable consumer item at an outlet to anactual draw since draw bounded sales data as opposed to adjusted nondraw bounded sales data compensating for hidden demand at occurrences ofsellouts inherently prejudices against a recommended draw as will nowbecome clear by way of the example below. In point of fact, there arefive mutually exclusive and exhaustive cases which may possibly arisedepending on the values of a recommended draw, an actual draw and salesat each time point which each impact various performance metricsdifferently as summarized in the table of FIG. 7.

Assuming that the draw of a newspaper at an outlet was 30 copies ofwhich 29 were sold, a recommended draw of 33 copies would be unfavorablybut fairly recorded with four returns as compared to the actual returnof a single copy. But, if all 30 copies were sold, the recommended drawof 33 copies would be unfavorably recorded with three returns but thistime unfairly since it belies the fact that in all likelihood at leastone and possibly all three of the three returns would have been sold.Assuming that the hidden demand at the occurrence of the selloutoccurring when all 30 copies were sold was, say, 2 copies, then therecommended draw of 33 copies would be unfavorably recorded with justone return but would be credited with two additional sold copies. Butassuming that the hidden demand at the occurrence of the sellout was 4copies, then the recommended draw of 33 copies would be unfavorablyrecorded with a sellout but with an additional three sold copies to itscredit.

To summarize, the present invention facilitates improved forecasting atboth at bottom level nodes and at higher level nodes, for example, say,at the level of a regional distributor. Moreover, the present inventionprovides for new more accurate performance evaluation techniquestogether with new performance metrics for evaluating an actual draw andfor comparing a recommended draw to an actual draw.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to better understand the invention and to see how it can becared out in practice, a preferred embodiment will now be described, byway of a non-limiting example only, with reference to the accompanyingdrawings in which:

FIG. 1 is a pictorial representation showing a demand forecast tree forcomputing demand forecast information for five different perishableconsumer items;

FIG. 2 is a table showing historical sales data associated with thedemand forecast tree of FIG. 1;

FIG. 3 is a pictorial representation showing a computer implementedsystem for implementing the present invention;

FIG. 4 is a flow char showing the steps for calculating the hiddendemand for a consumer item at an outlet at an occurrence of a sellout inaccordance with the present invention;

FIG. 5 is a graph showing the hidden demand for a newspaper atoccurrences of sellouts over a month of sales as estimated in accordancewith the present invention;

FIG. 6 is a graph comparing the efficacy of an adjusted demand forecastagainst sales for the newspaper with the efficacy of an unadjusteddemand forecast against sales for the newspaper for the same period of amonth of sales;

FIG. 7 is a table summarizing the five mutually exclusive and exhaustivecases which arise depending on the values of a recommended draw, anactual draw and sales for a perishable consumer item at an outlet forcomparing the efficacy of a recommended draw against the efficacy of anactual draw;

FIG. 8 is a graph showing a recommended draw for a newspaper forcomparison with an actual draw and sales; and

FIGS. 9A-9C is a table enabling the comparison of the efficacy of arecommended draw for a newspaper for a month to that of the actual drawfor the newspaper over the same month.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 shows a demand forecast tree 1 having a single top level node(00) with five branches A, B, C, D and E for correspondinglyrepresenting the sale of Item I (top level-1 node (10)) at Locations 1and 2 (bottom level nodes (11) and (21)), Item II (top level-1 node(20)) at Locations 1 and 3 (bottom level nodes (21) and (23)), Item III(top level-1 node (30)) at Location 1, 2 and 3 (bottom level nodes (31),(32) and (33)), Item IV (top level-1 node (40)) also at Locations 1, 2and 3 (bottom level nodes (41), (42) and (43)); and Item V (top level-1node (50)) at Location 1 (bottom level node (51)) only. FIG. 2 shows anexemplary table 2 containing historical sales data of Item I at thebottom level nodes (11) and (12). Similar tables exist for the sale ofthe other items at their respective outlets.

FIG. 3 shows a computer implemented system 3 as illustrated anddescribed in commonly assigned co-pending U.S. patent application Ser.No. 10/058,830 entitled “Computer Implemented Method and System forDemand Forecast Applications”, the contents are which are incorporatedherein by reference, capable of implementing the present invention. Thecomputer implemented system 3 includes a database server 4 for storingtime series of sales values, a forecast engine 6 including two or morecomputer servers 7 each independently capable of computing demandforecast information for an entire branch of the demand forecast tree 1,and a computer manager 8 for allocating branches into tasks.

The evaluation of the efficacy of a distribution policy for a perishableconsumer item, say, a newspaper, at an outlet over an evaluation period,say, a month, in accordance with the present invention is now describedwith reference to FIGS. 4 and 5. Sellouts of the newspaper at the outletover the evaluation period are identified by comparing the number ofsold copies (S) against draw (D) respectively shown as the solid graphline and the dashed graph line in FIG. 5. The sales values excludingtruncated sales values are input to a forecast engine, for example, theDemantra™ Demand Planner, commercially available from Demantra Ltd,Israel, to estimate mean demand values typically rounded to the nearestinteger for the newspaper at the sellouts over the month shown assquares in FIG. 5. The forecasted mean demand value at an occurrence ofa sellout may be greater than, equal to or less than the sales at thesellout. For example, the forecasted mean demand value for Day 5 is 29copies which is less than the draw of 33 copies whilst the forecastedmean demand value for Day 10 is 30 copies which is greater than thesales of 28 copies.

Based on the assumption that demand for the newspaper is a randomvariable X with a Poisson conditional probability distributionconditioned on the forecasted mean demand valueλ, the hidden demand H ateach sellout is estimated using the following expression

$ {H =  {E( {X - D} )} \middle| {X \geq D} } ) = {{\lambda( {1 + \frac{f(D)}{1 - {F(D)}}} )} - D}$For example, the hidden demand for Day 5 is 4 after having been roundedto the nearest integer based on the forecasted mean demand value λ=29and the draw D=33. The expected full demand EFD for the newspaper thatcould be expected to be sold at the outlet are then calculated at eachoccurrence of a sellout as follows: EFD=D+H (see crosses in FIG. 5).Continuing the above example, the expected full demand for Day 5 is33+4=37 copies. The performance metrics for evaluating the efficacy ofthe distribution policy for the newspaper at the outlet over theevaluation period can include inter alia: the total hidden demand forthe newspaper at the outlet over the evaluation period; the ratio oftotal hidden demand to total sales over the evaluation period; and theratio of total hidden demand to draw over the evaluation period. In thepresent case, the total hidden demand for the newspaper is 19 copieswhich is equal to 2.12% of sales and 1.90% of draw.

The adjusted sales data can be beneficially employed to improve demandforecasting as evidenced in FIG. 6 in which an adjusted demand forecastderived from adjusted sales data yields a reduced MAPE=8.79% as opposedto the MAPE=12.10% of an unadjusted demand forecast derived fromunadjusted sales data where:

${MAPE} = {\frac{\Sigma{{{{Actual}\mspace{14mu}{Sales}\mspace{14mu}{Values}} - {{Demand}\mspace{14mu}{Forcast}\mspace{14mu}{Values}}}}}{\sum| {{Actual}\mspace{14mu}{Sales}\mspace{14mu}{Values}} } \times 100\%}$with the summation being over all the time points over a given period.

The evaluation of the efficacy of a recommended draw for a perishableconsumer item, say, a newspaper, at an outlet to an actual draw over anevaluation period, say, a month, in accordance with the presentinvention is now described with reference to FIGS. 7 to 9. The entriesin the different columns of the table of FIG. 7 are derived from thefollowing relationships:ES=min(S+H,RD)ER=RD−ESESO=δ(H+S/RD)EST=ESO*(S+H≧RD)BS=min(S,RD)BR=RD−BS=max(0,RD−S)BSO=δ(RD≦S)BST=max(0,S−RD)The four bias columns of FIG. 7 show the biases in four performancemetrics ES-BS, ER-BR, ESO-BSO and EST-BST which inherently exist byfailing to take into account hidden demand at occurrences of sellouts,as particularly applies to Case 4. The four Δcolumns of FIG. 7 show thegains or losses in four performance metrics ES-S, ER-R, ESO-So andEST-ST by virtue of delivery of a consumer item to an outlet inaccordance with a recommended draw as opposed to an actual draw.

Turning now to FIGS. 8 and 9, the recommended draw (shown as a dashedline in FIG. 8) is preferably arrived at by initially running a forecastengine on historical adjusted sales data to estimate a demand forecastfor the newspaper at the outlet. The hidden demand at the occurrences ofsellouts is preferably estimated as described hereinabove with referenceto FIGS. 4 and 5. Thereafter, safety stock is added to the demandforecast at each time point over the evaluation period in accordancewith a predetermined availability percentage, say, 80%, typicallyrounded to the nearest integer to yield the recommended draw.

The month evaluation period includes five occurrences of sellouts (shownas circles in FIG. 7) on Days, 1, 9, 19, 21, and 28, namely, 16.13%sellouts, with total sales of 876 out of a total draw of 985 copies,namely, 11.07% return. The present invention estimates that the hiddendemand at the occurrences of the sellouts on Days, 1, 9, 19, 21, and 28is as follows: 5, 4, 5, 4, and 4, respectively, making for a totalstockout of 22 copies which is the equivalent of 2.51% total sales and2.23% total draw. The recommended draw would result in a lower draw of960 copies as opposed to the actual draw of 985 copies but with onlyfour occurrences of sellouts on Days 2, 8, 21, and 28. Not only is therecommended draw 2.54% lower than the actual draw but the recommendeddraw on taking into consideration the hidden demand at occurrences ofsellout would achieve 1.03% higher expected sales of 885 copies asopposed to actual sales of 876 copies and as opposed to draw boundedsales of just 871 copies if the hidden demand at occurrences selloutswould not have been taken into account. The recommended draw on takinginto account the hidden demand at the occurrences of sellout would beexpected to render 12.90% sellouts and 7.81% returns with Column ES-Rand ES-R in FIG. 9 showing the gained sales and reduced returnachievable by the recommended draw in comparison to the actual draw.

While the invention has been described with respect to a limited numberof embodiments, it will be appreciated that many variations,modifications, and other applications of the invention can be madewithin the scope of the appended claims. Other inferior approaches maybe employed for arriving at mean demand values not requiring a run of aforecast engine, for example, assuming that a mean demand value for aperishable consumer item at an outlet at an occurrence of a sellout isthe truncated sales value.

1. A computer-implemented method for estimating the hidden demand for aperishable consumer item at an outlet at an occurrence of a sellout foruse with a demand forecast tree having at least one node with a timeseries of sales values associated therewith representing the actualsales of the perishable consumer item at the outlet over an observationperiod, the observation period comprising at least one occurrence of asellout, the method comprising: determining a subset of sales values ofthe time series of actual sales values over the observation period forthe perishable consumer item at the outlet, the subset of sales valuesexcluding the actual sales value(s) at the at least one occurrence of asellout, the occurrence of the sellout being determined by comparing asales value of the time series of sales values against a correspondingdraw quantity of a time series of draw quantities; applying, using acomputer, a statistical seasonal causal time series forecasting model ofcount data on the subset of sales values to determine a forecasted meandemand value for the perishable consumer item at the outlet at theoccurrence of the sellout; and estimating the hidden demand at theoccurrence of the sellout using a single parameter probabilitydistribution conditioned on the forecasted mean demand value, whereinthe forecasted mean demand value is calculated from the subset of actualsales values excluding the actual sales value(s) at the at least oneoccurrence of the sellout; and wherein the single parameter probabilitydistribution is conditioned on the forecasted mean demand value.
 2. Themethod according to claim 1 wherein the single parameter probabilitydistribution is Poisson.
 3. The method according to claim 1 wherein thesubset of sales values excludes the actual sales values at alloccurrences of sellouts over the observation period.
 4. The methodaccording to claim 1 and further comprising: calculating the value of atleast one performance metric on the basis of adjusted sales datacompensating for hidden demand at occurrences of sellouts over anevaluation period.
 5. The method according to claim 4 wherein the stepof calculating the value of at least one performance metric includescalculating the total stockout for the perishable consumer item at theoutlet over the evaluation period for evaluating the efficacy of adistribution policy for the perishable consumer item at the outlet overthe evaluation period.
 6. The method according to claim 4 wherein thestep of calculating the value of at least one performance metricincludes calculating the value of at least one performance metricrelating to the sale of the perishable consumer item at the outlet whichcould be expected to occur over the evaluation period by virtue of theperishable consumer item being delivered in accordance with arecommended distribution policy as opposed to an actual distributionpolicy for comparing the efficacy of the recommended distribution policyto the efficacy of the actual distribution policy over the evaluationperiod.
 7. The method according to claim 6 wherein the step ofcalculating the value of at least one performance metric includescalculating the value of at least one performance metric from thefollowing list of performance metrics: change in sales, change inreturns, change in number of sellouts, and change in stockout.
 8. Themethod according to claim 1 wherein the perishable consumer item is aprinted media publication.
 9. A computer-implemented system forestimating the hidden demand for a perishable consumer item at an outletat an occurrence of a sellout for use with a demand forecast tree havingat least one node with a time series of sales values associatedtherewith representing the actual sales of the perishable consumer itemat the outlet over an observation period, the observation periodcomprising at least one occurrence of a sellout, the system comprising:a database server for storing time series of sales values over anobservation period; a forecast engine server for computing demandforecast information for the demand forecast tree; and a processor forperforming the steps of: determining a subset of sales values of thetime series of actual sales values over the observation period for theperishable consumer item at the outlet, the new subset of sales valuesexcluding the actual sales value(s) at the at least one occurrence of asellout, the occurrence of the sellout being determined by comparing asales value of the time series of sales values against a correspondingdraw quantity of a time series of draw quantities; applying astatistical seasonal causal time series forecasting model of count dataon the subset of sales values to determine a forecasted mean demandvalue for the perishable consumer item at the outlet at the occurrenceof the sellout; and estimating the hidden demand at the occurrence ofthe sellout using a single parameter probability distributionconditioned on the forecasted mean demand value, wherein the forecastedmean demand value is calculated from the subset of actual sales valuesexcluding the actual sales value(s) at the at least one occurrence ofthe sellout; and wherein the single parameter probability distributionis conditioned on the forecasted mean demand value.
 10. The systemaccording to claim 9 wherein the single parameter conditionalprobability distribution is Poisson.
 11. The system according to claim 9wherein the subset of sales values excludes the actual sales values atall occurrences of sellouts over the observation period.
 12. The systemaccording to claim 9 and further comprising executing a step of:calculating the value of at least one performance metric on the basis ofadjusted sales data compensating for hidden demand at occurrences ofsellouts over an evaluation period.
 13. The system according to claim 12wherein the step of calculating the value of at least one performancemetric includes calculating the total stockout for the perishableconsumer item at the outlet over the evaluation period for evaluatingthe efficacy of a distribution policy for the perishable consumer itemat the outlet over the evaluation period.
 14. The system according toclaim 12 wherein the step of calculating the value of at least oneperformance metric includes calculating the value of at least oneperformance metric relating to the sale of the perishable consumer itemat the outlet which could be expected to occur over the evaluationperiod by virtue of the perishable consumer item being delivered to theoutlet in accordance with a recommended distribution policy as opposedto an actual distribution policy for comparing the efficacy of therecommended distribution policy to the efficacy of the actualdistribution policy over the evaluation period.
 15. The system accordingto claim 14 wherein the step of calculating the value of at least oneperformance metric includes calculating the value of at least oneperformance metric from the following list of performance metrics:change in sales, change in returns, change in number of sellouts, andchange in stockout.
 16. The system according to claim 9 wherein theperishable consumer item is a printed media publication.